Skip to content

fix(ml): reject float16 graph-I/O feeds with the artifact-contract error (#111 root cause) - #134

Merged
ronaldtse merged 1 commit into
mainfrom
fix/issue-111-zero-size-tensor
Nov 12, 2026
Merged

ronaldtse merged 1 commit into
mainfrom
fix/issue-111-zero-size-tensor

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

Root cause (issue #111)

The not enough space: expected N, got 0 failure from onnxruntime-node's
session.run is float16 graph I/O meeting the native binding — not a
windowing bug and not a memory/size problem.

The issue-era artifacts (fas-diac-base-1.0, urd-diac-base-1.0, released
2026-10-18) were fp16 exports whose graph boundaries stayed float16:
last_hidden_state, encoder_hidden_states, logits. Python ORT
consumes fp16 I/O natively, so the reference decoder — and the frozen
conformance vectors generated with it — never saw the condition; every
other runtime breaks (the models-repo WO79 finding,
tools/normalize_fp16_io.py's own docstring).

Mechanism, reproduced from a 1KB fixture on current HEAD

  1. The encoder's float16 output arrives in JS intact: on Node 24,
    last_hidden_state comes back as type=float16 dims=[1,4] data=Float16Array(4).
  2. The runtime feeds it back as encoder_hidden_states. ORT's JS
    Tensor constructor converts every float16 feed into a
    Float16Array view when the runtime has native Float16Array
    (Node >= 24) — onnxruntime-common tensor-impl.js (Uint16Array
    → Float16Array wrap).
  3. onnxruntime-node's N-API binding cannot read a Float16Array buffer —
    it sees byte length 0 — and run() fails:
    not enough space: expected 8, got 0 from the binding
    (lib/backend.ts:130, the exact stack location and signature from
    the issue).

The issue's own numbers confirm it

"expected" = dims product × the float16 element size of the
encoder_hidden_states feed, "got" = the feed's byte length (0):

  • fas: 288512 = 98 × 1472 × 2 — vector row 1 is 97 UTF-8 bytes →
    seq 98 (bytes + EOS); d_model 1472 (byt5-small); fp16 element size.
  • urd: 73728 = 24 × 1536 × 2 — vector row 1 is 23 bytes → seq 24;
    d_model 1536 (byt5-base); fp16 element size.

"got" is always the buffer length, never the artifact size — which is
why the failure is size-independent (711MB and 1.35GB artifacts fail
identically).

Node-version dependence (verified empirically)

runtime Float16Array float16 feed result
Node 20.20.2 undefined Uint16Array-backed — decodes fine
Node 22.20.0 undefined Uint16Array-backed — decodes fine
Node 24.14.0 native Float16Array view — binding reads 0 bytes, opaque error
Node 26.5.0 native same — opaque error

Why it stopped reproducing — and why it is still live

  • Artifact side (fixed upstream 2026-10-21): interscript-models
    WO79 (#327) shipped tools/normalize_fp16_io.py and replaced both
    release zips in place with fp32-graph-I/O repacks. Both frozen
    conformance legs pass byte-exactly today (fas 12/12, urd 12/12,
    re-verified on this branch).
  • Still live in the runtime: there was no defense. Any future
    artifact exported with fp16 graph I/O crashes on Node >= 24 with the
    opaque binding error, and CI cannot see it — the tiny fixture zips
    are fp32-I/O (test/fixtures/tiny-imf.zip). The issue's original
    windowing suspicion (windows.ts) is exonerated: the failure
    reproduces on a 3-char input through the single-window
    translateDirect path, with no window split involved.

The fix

Minimal, at the binding boundary where the condition is known
(src/ml/session/onnx-node.ts):

  • When the backend would hand onnxruntime-node a float16 feed on a
    runtime where the binding cannot consume it, reject it with the
    artifact-contract error — naming the offending input and the
    re-export path (tools/normalize_fp16_io.py) — instead of the
    opaque binding failure.
  • Tensor.type / TensorData now include float16 / Uint16Array:
    fp16 outputs have been flowing through these paths via an unsound
    cast; the types now say what actually flows.

No decode-path behavior change: fp32-I/O artifacts (all shipped ones)
never construct a float16 feed, so the guard is a no-op for them; the
browser (WASM) backend handles fp16 feeds itself and is untouched.

Regression coverage

  • test/fixtures/tiny-fp16io.zip (~1KB): tiny-imf.zip with float16
    Cast-wrapped across both graph boundaries (encoder output, decoder
    input, decoder output) and recomputed member digests — the pre-WO79
    export shape, CI-loadable.
  • test/imf-fp16io.test.ts locks both behaviors. Red first: against
    the unguarded base it failed with exactly
    got 'not enough space: expected 8, got 0'. Green after the fix on
    Node 24 (rejection branch) and on Node 22.20.0 (decode branch).

Verification

  • test/imf-fp16io.test.ts: green on Node 24.14.0 and Node 22.20.0
  • fas-diac-base-1.0-fp16 conformance leg: 12/12 vectors byte-exact
    post-fix (226.9s); urd leg 12/12 (106.5s, base commit)
  • full npx vitest run: 352 passed; the only failing file is
    test/cli.test.ts (13 tests) — verified failing identically at the
    base commit (2186ebe) in a clean base worktree, pre-existing and
    unrelated
  • tsc --noEmit, eslint, prettier clean

Closes #111 (the shipped artifacts already conform; this closes the
runtime-side hole and locks the contract with a CI-loadable fixture).

Fixes #111.

Issue #111's "not enough space: expected N, got 0" from
onnxruntime-node's session.run is float16 graph I/O meeting the
binding, not a windowing or size bug:

- The issue-era fp16 exports shipped float16 graph boundaries
  (last_hidden_state / encoder_hidden_states / logits). Python ORT
  consumes fp16 I/O natively, so the reference decoder and the frozen
  vectors never saw it (the models-repo WO79 finding).
- On runtimes with native Float16Array (Node >= 24), ORT's JS Tensor
  converts every float16 feed into a Float16Array view whose bytes the
  N-API binding reads as zero, and run() fails with the opaque error.
  The issue's numbers decode exactly as seq x d_model x 2 bytes of
  feed with a 0-byte buffer: fas 288512 = 98 x 1472 (byt5-small) x 2,
  urd 73728 = 24 x 1536 (byt5-base) x 2. Size-independent because
  "got" is always the buffer length, not the artifact.
- Node 20/22 (no Float16Array) consume the same feed as Uint16Array
  and decode fine: verified 20.20.2/22.20.0 ok, 24.14.0/26.5.0 broken.

The shipped artifacts have been IO-normalized since 2026-10-21
(models-repo WO79 #327 replaced both release zips), so both conformance
legs pass today; the runtime had no defense against the next fp16-IO
export and CI cannot see it (the tiny fixtures are fp32-I/O).

Fix: reject float16 feeds at the onnx-node binding boundary with the
artifact-contract error (input name + normalize_fp16_io.py pointer)
instead of the opaque binding failure, and type Tensor honestly for
what already flows (float16 / Uint16Array). fp32-I/O artifacts never
construct a float16 feed, so the guard is a no-op for them; the
browser (WASM) backend handles fp16 feeds itself and is untouched.

Regression: test/fixtures/tiny-fp16io.zip (~1KB, the pre-WO79 export
shape Cast-wrapped from tiny-imf.zip with recomputed member digests)
plus test/imf-fp16io.test.ts, which reproduced the opaque binding
error before the fix and locks both runtime branches (decode on Node
20/22, explicit rejection on Node 24+).
@ronaldtse
ronaldtse merged commit 44695b2 into main Nov 12, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TS IMF runtime fails the frozen conformance fixtures on real artifacts (zero-size tensor in windowed decode)

1 participant